Papers with RC models
Break, Perturb, Build: Automatic Perturbation of Reasoning Paths Through Question Decomposition (2022.tacl-1)
Copied to clipboard
| Challenge: | Recent efforts to create challenge benchmarks that test the abilities of natural language understanding models have largely depended on human annotations. |
| Approach: | They propose a framework for automatic reasoning-oriented perturbation of question-answer pairs that decomposes a question into reasoning steps required to answer it and generates new question-anchor pairs. |
| Outcome: | The proposed framework generates evaluation sets for reading comprehension benchmarks and generates examples without human intervention. |
Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension (2020.tacl-1)
Copied to clipboard
| Challenge: | Innovations in annotation methodologies have been a catalyst for Reading Comprehension (RC) datasets and models. |
| Approach: | They propose to use a model-in-the-annotation-loop approach to train adversarial models in three different settings to explore reproducibility of the adversarial effect, transfer from data collected with varying model- in-the loop strengths, and generalization to data collected without a modeling model. |
| Outcome: | The proposed approach achieves 39.9F1 on questions it cannot answer when trained on SQUAD, but lower than when trained using RoBERTa itself (41.0F1). |
NUT-RC: Noisy User-generated Text-oriented Reading Comprehension (2020.coling-main)
Copied to clipboard
| Challenge: | Existing RC models focus on extractive or generative, but ignore integration of them. |
| Approach: | They propose a noisy user-generated text-oriented RC model that integrates extractive and generative RC models by a multi-task learning mechanism and an answer selection module. |
| Outcome: | The proposed model outperforms state-of-the-art models on Twitter. |
Connecting Attributions and QA Model Behavior on Realistic Counterfactuals (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent research in interpretability of neural models has yielded numerous token attribution techniques, but it is hard to evaluate whether these explanations are faithful. |
| Approach: | They propose to use pairwise attributions to connect outputs to high-level model behavior to examine how well different attribution techniques align with this assumption on realistic counterfactuals in the case of reading comprehension (RC). |
| Outcome: | The proposed methods are better suited to RC than token-level attributions across different RC settings, and the best performance comes from a modification that was proposed to an existing pairwise attribution method. |
CORE: A Few-Shot Company Relation Classification Dataset for Robust Domain Adaptation. (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets that focus on company relations and business entities are lacking in relation classification. |
| Approach: | They introduce a few-shot relation classification dataset for company relations and business entities . they use a dataset that includes 4,708 instances of 12 relation types . |
| Outcome: | The proposed dataset includes 4,708 instances of 12 relation types with corresponding textual evidence extracted from company Wikipedia pages. |